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Community colleges play a vital role in postsecondary education, enrolling more than one in every 
three postsecondary students. White their importance has grown over the past 50 years, their stu¬ 
dents’ success rates remain low. Consequently, community college stakeholders are searching with 
mounting urgency for approaches that increase students 'success rates. “Learning communities ” are 
one popular strategy. This article describes the results of one of the largest (n ~ 7,000) independent 
randomized trials in higher education history—an evaluation of the effectiveness of learning com¬ 
munities for students in developmental education at six community colleges throughout the United 
States. The estimated overall average effects of learning communities are positive, although quite 
modest. 

Keywords: learning community, learning communities, community college, randomized trial, pro¬ 
gram evaluation 


Over the last five decades, community col¬ 
leges have played an increasingly large role in 
American postsecondary education. In 1963, 
community colleges enrolled 740,000 students, 
representing 15% of all fall enrollment in post¬ 
secondary institutions (Provasnik & Planty, 
2008). By 2011, enrollment had increased by 
nearly 1,000%, to 7.1 million students, repre¬ 
senting 34% of all fall enrollment (Snyder & 
Dillow, 2013). This shift in the postsecondary 
landscape is due in part to the fact that commu¬ 
nity colleges are generally open-access institu¬ 
tions and are typically lower cost than 4-year 
colleges and universities. Tuition and fees 
at community colleges are around one third 
of their cost at public 4-year colleges and 


universities, and around one tenth of their cost 
at private 4-year colleges and universities (The 
College Board, 2012). 

While community colleges have provided 
increased access to many students, particularly 
nontraditional, low-income, and minority stu¬ 
dents, the completion rates at these institutions 
leave much room for improvement. Among stu¬ 
dents beginning at public 2-year colleges, fewer 
than half earn a credential or transfer to a 4-year 
institution within 6 years after their initial enroll¬ 
ment (Radford, Berkner, Wheeless, & Shepherd, 
2010 ). 

One factor contributing to these low success 
rates is that many students arrive on campus 
academically unprepared for college-level 
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coursework (Conley, 2010; Long & Boatman, 
2013). One estimate of the prevalence of the 
need for developmental courses suggests that 
among students whose first institution attended 
was a community college, more than 60% took 
a remedial course at a postsecondary institution 
(Adelman, 2004). Unsurprisingly, degree or 
certificate attainment rates among students who 
need developmental education are even lower 
than that of the general population (Attewell, 
Lavin, Domina, & Levey, 2006). Students in 
need of developmental education are not just 
failing to earn a credential—the majority never 
complete the developmental course sequence to 
which they are referred (Bailey, Jeong, & Cho, 
2010 ). 

Prompted by these statistics, community col¬ 
lege stakeholders are searching for approaches 
that increase students’ chances of earning a col¬ 
lege credential—particularly for students referred 
to developmental education courses. “Learning 
communities” are one of the most popular strate¬ 
gies to help such students. 1 

Learning Communities 

Although definitions of learning communities 
vary, there is much agreement about their key ele¬ 
ments. Learning communities involve the coen¬ 
rollment of a cohort of students into two or more 
courses. Typically, the curricula of these courses 
are intentionally linked or integrated, sometimes 
around a theme. For example, a learning commu¬ 
nity with an overarching theme and title called 
“Poverty and Inequality” might link a develop¬ 
mental English course and a college-level sociol¬ 
ogy course. The courses could require that 
students learn various elements of essay writing 
and argumentative rhetoric in their English course 
and then use those techniques to write a paper in 
their sociology course exploring the relationship 
between income status and race. Learning com¬ 
munities tend to require instructor collaboration, 
which enables teachers to communicate about 
their shared students and to integrate the curricu¬ 
lum across courses through, for example, joint 
assignments and readings. Some definitions of 
learning communities also include a pedagogical 
component, usually focusing on “active engage¬ 
ment,” “active pedagogy,” and/or “collaborative 
learning.” Finally, some learning communities 


include add-on student supports, such as enhanced 
counseling, tutoring, or other student services 
(Minkler, 2002; Richburg-Hayes, Visher, & 
Bloom, 2008; Smith, 2001; Smith, MacGregor, 
Matthews, & Gabelnick, 2004). 

In practice, learning communities vary from 
extremely basic (e.g., simply coenrolling stu¬ 
dents in two or more classes) to multifaceted 
(e.g., coenrollment, tightly integrated curriculum 
including joint assignments and assessments, 
strong faculty collaboration, and additional stu¬ 
dent support services). While there is a signifi¬ 
cant gap between idealized learning communities 
and many enacted learning communities, some 
form of learning communities operate at an esti¬ 
mated 400 or 500 colleges and universities 
throughout the United States (Smith, 2001; Smith 
et al., 2004). At community colleges, a national 
study reported that around half of the 288 col¬ 
leges surveyed were operating at least one learn¬ 
ing community (Center for Community College 
Student Engagement, 2012). 

Why Learning Communities Are Expected 
to Work 

Learning communities at community colleges 
have the potential to improve student outcomes 
through several mechanisms described briefly 
here. In Tinto’s (1975) framework on dropout 
from higher education, he posits, 

It is the person’s normative and structural integration 
into the academic and social systems that lead to new 
levels of commitment. Other things being equal, the 
higher the degree of integration of the individual into 
the college systems, the greater will be his 
commitment to the specific institution and to the goal 
of college completion, (p. 96) 

Yet, community colleges are attended largely 
by commuter students, many of whom contend 
with the competing demands of work, family 
obligations, and school (Brock & LeBlanc, 
2005; Minkler, 2002). Consequently, for many 
students the only time they are on campus is 
when they attend class, limiting opportunities 
for integration. Some critics of Tinto’s model in 
the community college context claim that for 
2-year-college students, particularly commuting 
students, demographics and external circum¬ 
stances are the primary factors in dropout, rather 
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than integration on campus (Bean & Metzner, 
1985; Braxton, Sullivan, & Johnson, 1997). 
Critics note that for community college students, 
particularly students of color, Tinto’s idea of dis¬ 
connecting from a previous community and con¬ 
necting to a new college community is not 
applicable, as community college students typi¬ 
cally attend while still in their home communi¬ 
ties (Guiffrida, 2006). Therefore, the benefits of 
learning communities may accrue differently for 
students of different backgrounds. Recent work, 
however, finds that community college students, 
including students of color, experience a type of 
interconnected social and academic integration, 
which, while distinct from that of undergraduates 
at 4-year, residential universities, does play a role 
in students’ persistence and success (Deil-Amen, 
2011; Karp, Hughes, & O’Gara, 2010). 

By coenrolling students into two or more 
courses, learning communities are intended to 
foster stronger connections among students 
(Smith et al., 2004). Through sustained academic 
relationships among students and faculty, stu¬ 
dents are expected to feel more integrated into a 
community of peers and college life, leading to a 
greater level of commitment. In addition, inte¬ 
grating course materials may help students 
understand connections between disciplines and 
between what they are learning in school and 
their personal lives, and in doing so, engage stu¬ 
dents more deeply in learning. Curricular inte¬ 
gration may be particularly effective when a 
developmental-level course and college-level 
course are paired in a learning community as this 
allows students to use the basic skills being 
taught in developmental classes in their college- 
level coursework. Broadly speaking, learning 
communities are theorized to lead to improved 
academic outcomes by fostering stronger con¬ 
nections among students and between students 
and faculty, integrating students into campus life, 
and providing a more engaging academic envi¬ 
ronment. This is hypothesized to improve stu¬ 
dents’ academic attainment and increase their 
likelihood of persisting in school. 

Learning communities are a popular strategy 
for teaching students in need of developmental 
education and increasing their likelihood of mov¬ 
ing on to college-level coursework. The social 
integration encouraged by coenrollment in mul¬ 
tiple classes can be extremely important for 


academically underprepared students, who may 
be more marginalized from the campus commu¬ 
nity and have less campus knowledge (Person, 
Rosenbaum, & Deil-Amen, 2006). Moreover, the 
connection between linked developmental 
courses and college-level courses may bolster 
learning in each course (Rutschow & Schneider, 
2011 ). 

Evidence of the Effectiveness of Learning 
Communities 

Over the last two decades, there has been a 
significant amount of research suggesting that 
learning communities have positive effects on 
students. Among the first of the studies published 
in a peer-reviewed journal was a study by Tinto 
and Russo (1994) of a Coordinated Studies 
Program (CSP) that found that students who par¬ 
ticipated in the CSP increased involvement in 
academic and social activities, reported more 
positive views of college, and persisted to the 
following semester at a higher rate. 

Subsequent to Tinto and Russo’s findings, 
multiple studies of a variety of different learning 
communities tended to support the finding that 
learning communities have a positive effect on 
student engagement, retention, and academic 
progress (Engstrom & Tinto, 2008; Goldberg & 
Finkelstein, 2002; Gordon, Young, & Kalianov, 
2001; Stassen, 2003; Taylor, Moore, MacGregor, 
& Lindblad, 2003; Tinto, 1997, 1998; Wilmer, 
2009; Zhao & Kuh, 2004). These results were 
generally quite promising; however, the studies 
all had major limitations with respect to their 
designs’ ability to make causal claims about the 
effectiveness of learning communities. 

A key limitation in all but one of these studies 
was the way in which a comparison group was 
selected. Generally, comparisons were made 
between students who were in learning communi¬ 
ties and those who were not, despite the fact that 
students typically self-select into learning com¬ 
munities and are unlikely equivalent. Two studies 
controlled for a limited set of measured back¬ 
ground characteristics in their statistical models, 
such as gender, race, and high school SAT/ACT 
scores; however, the students who chose to enroll 
in learning communities may still have differed 
from those who did not in other unmeasured 
ways, such as tenacity and motivation. With 
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respect to establishing the causal effect of learn¬ 
ing communities, the designs used in these past 
studies were quite limited, and none conducted 
analyses to determine the sensitivity of their find¬ 
ings to selection bias. 

To our best knowledge, the first study not sub¬ 
ject to selection bias was a random assignment 
evaluation of learning communities by Goldberg 
and Finkelstein (2002). They reported on an 
evaluation that randomly assigned 25 full-time 
Electronic Technician Certificate students—16 
were randomly assigned to a team-taught learn¬ 
ing community and 9 were randomly assigned to 
individually taught, unlinked classes. The study 
found positive impacts on student perceptions of 
academic and social integration but no impacts 
on behavioral outcomes, such as grade point 
averages or persistence to the second semester. 
This could be a result of lack of program effects, 
but it could also easily be a result of the lack of 
statistical power to detect meaningful impacts 
with such a small sample size. 

The first large sample random assignment 
evaluation of learning communities was con¬ 
ducted by MDRC in 2003 to 2005 to evaluate the 
effectiveness of Kingsborough Community 
College’s (KCC) learning community program. 
Evaluators randomly assigned more than 1,500 
students to either a program group, which had the 
opportunity to participate in the Opening Doors 
Learning Community, or a control group, which 
could enroll in the colleges’ usual unlinked 
courses. More than two thirds of the sample 
required developmental English at the start of the 
study (and the learning communities to which 
they were assigned included a developmental 
English course as one of the linked courses). 
Overall, the program had positive effects on stu¬ 
dents’ engagement, performance in the targeted 
developmental English course, and total number 
of credits earned 2 years after the study began. 
The evidence of positive impacts on persistence 
was less promising, with no increased persistence 
in the main semester after the learning commu¬ 
nity ended but some evidence of an effect three 
semesters after the program ended (Richburg- 
Hayes et al., 2008; Scrivener et al., 2008). 

In part as a result of the positive early findings 
from the evaluation at KCC, MDRC launched the 
Learning Communities Demonstration—a large- 
scale evaluation funded through the National 


Center for Postsecondary Research of the effec¬ 
tiveness of six learning community models, each 
operated at a community college in the United 
States. 2 The demonstration was designed to obtain 
separate unbiased impact estimates at each col¬ 
lege, each with enough precision that relatively 
small impacts could be detected with high confi¬ 
dence (sample sizes were more than 1,000 per 
college, yielding minimum detectable effect sizes 
below .20 at each college). This article synthe¬ 
sizes the results from this group of experiments, 
focusing on the effectiveness of learning commu¬ 
nity models that target students in developmental 
education. This synthesis represents one of the 
largest (n ~ 7,000) independent random assign¬ 
ment evaluations in higher education history. 

Objectives 

This article examines the pooled overall aver¬ 
age effect of six learning community programs on 
academic progress. The study participants were 
all academically underprepared for college-level 
coursework in either English or math. This means 
that they scored below a cut score, set by each 
college, on the colleges’ placement exam. In addi¬ 
tion, the study investigates whether the effects of 
the six learning community programs vary sig¬ 
nificantly. That is, were the effects of these pro¬ 
grams fairly homogeneous or were they different 
from college to college? As described in more 
detail below (and in past reports), the six colleges 
operated varying learning community models, 
they varied in their implementation of those mod¬ 
els, they varied in their contexts, and they varied 
with respect to the types of students they serve; 
consequently, it is possible that the effects of 
learning communities might be large at some col¬ 
leges and small or null at other colleges. 

Because the six programs all operated in real- 
world school-based settings, compliance with the 
treatment was imperfect. While almost no con¬ 
trol group members crossed over and experi¬ 
enced learning communities (less than 1%), 
around 29% of program group members were 
no-shows, that is, they were not enrolled in a 
learning community at the deadline for students 
to add or drop courses, despite being randomly 
assigned the opportunity to enroll in a learning 
community. This article focuses on the effect of 
the intent-to-treat (ITT), which includes these 
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no-shows and thus provides an estimate of the 
effect of offering learning communities in a real- 
world school-based setting, where compliance is 
imperfect. Sensitivity analyses, which more 
closely approximate the effect of treatment-on- 
the-treated (TOT), are also described. 

Method 

Sample 

The studied learning community programs are 
located at six community colleges throughout the 
continental United States: The Community 
College of Baltimore County (CCBC) in 
Maryland, Hillsborough Community College 
(Hillsborough) in Florida, Houston Community 
College (Houston) in Texas, KCC in New York, 
Merced College (Merced) in California, and 
Queensborough Community College (QCC) in 
New York. The institutions were selected by 
MDRC after a national search for community 
colleges that would reflect a range of experiences 
among students enrolled in developmental edu¬ 
cation learning communities across the country. 
Candidates were identified through networks and 
individuals very familiar with community col¬ 
leges, including professional associations, pro¬ 
fessional development providers, and research 
organizations. MDRC conducted dozens of 
phone meetings and site visits to candidate col¬ 
leges to assess whether colleges met the key cri¬ 
teria for selection. Criteria for selection included 
the requirement that each college exhibit a rea¬ 
sonably strong contrast between learning com¬ 
munities and standard developmental education 
classes; offered learning communities that coen¬ 
rolled students in at least two courses and 
included at least some cross-curricular instruc¬ 
tion; offered a core developmental subject (e.g., 
math or English) across the learning communi¬ 
ties; demonstrated sufficient instructor and stu¬ 
dent interest in the program to meet study intake 
goals and to make random assignment possible; 
and agreed to cooperate with the researchers to 
facilitate data collection and study procedures. 
The original intent was to select a purposive sam¬ 
ple among colleges that met these criteria, repre¬ 
senting the range of developmental education 
learning communities in operation in community 
colleges, which tend to vary from what Stassen 
(2003) referred to as “least coordinated” to “most 


connected.” While individual examples of 
advanced learning communities were found 
within some colleges, it proved difficult to iden¬ 
tify colleges that offered programs where this 
was the norm across most learning communities. 
Ultimately, the six colleges presented in this 
study represent a range of developmental educa¬ 
tion learning communities. None of the colleges 
had only learning communities of the “most con¬ 
nected” type, although all of the colleges had at 
least one or two. This article provides some 
information about the recruitment effort, the spe¬ 
cific colleges, their learning community models, 
and their implementation; for more detailed 
accounts, see D. Bloom and Sommo (2005); 
Richburg-Hayes et al. (2008); Scrivener et al. 
(2008); Visher, Schneider, Wathington, and Collado 
(2010); Visher and Teres (2011); Weiss, Visher, 
and Wathington (2010); Weissman et al. (2011); 
and Weissman, Cullinan, Cema, Safran, and 
Richman (2012). 

At five of the six colleges described in this 
article, the studied learning community programs 
targeted students in need of developmental educa¬ 
tion only, rather than the colleges’ entire student 
body, and provided students with learning com¬ 
munities that included both developmental and 
college-level courses. Only at KCC were “college- 
ready” students, defined as those needing no 
developmental reading/writing courses, also 
included in the eligible population. This synthesis 
focuses on the effectiveness of learning communi¬ 
ties that include a developmental course as part of 
the learning community; consequently, in this 
article KCC’s sample includes only those students 
with developmental English needs. 3 Four of the 
six programs targeted students in need of develop¬ 
mental English (CCBC, Hillsborough, KCC, and 
Merced), whereas the other two programs targeted 
students in need of developmental math (Houston 
and QCC). A brief overview of each program 
model is provided in the first column of Table 1. A 
description of students’ characteristics is provided 
in the following section. 

Random Assignment Procedure and Student 
Characteristics 

College staff invited students who had tested 
in developmental English or math to participate 
in the learning communities evaluation through 
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TABLE 1 

Overview of Developmental Education Learning Communities, by College 


Learning community program model Summary of key implementation findings 

The Community College of Baltimore County (Baltimore, MD) 


Developmental reading or writing linked with a 
college-level course 

A faculty “Master Learner” sat in the college- 
level course and conducted a weekly, 1-hour, 
noncredit seminar on learning-to-learn in the 
context of that course 

Hillsborough Community College (Tampa, FL) 
Developmental reading linked with a student 
success course 

Success course focused on acclimation to 
college and study skills 

Houston Community College (Houston, TX) 
Developmental math linked with a student 
success course 

Success course focused on acclimation to 
college and study skills 

Kingsborough Community College (Brooklyn, NY) 
Developmental English linked with a college- 
level course in the student’s major and a one- 
credit freshman orientation course 
Program included enhanced advising, tutoring, 
and a textbook voucher 

Merced College (Merced, CA) 

Developmental writing linked with 
developmental reading or math, a college-level 
course, or a student success course 
Links included cross-content themes and 
integrated assignments 

Queensborough Community College (Queens, NY) 
Developmental math linked with developmental 
or college-level English (fall 2007) or with a 
college-level course 


There was a significant variation in implementation 
among the learning communities in the program, 
particularly in the areas of curricular integration and 
faculty collaboration. The Master Learner content also 
varied considerably by instructor 


Curricular integration was generally not emphasized by 
program leaders, although by the last semester of the 
evaluation, some instructors were attempting to align 
and integrate their courses 

Curricular integration was not initially emphasized by 
program leaders, but as the program evolved, most 
instructors worked with each other to create joint 
assignments 

The learning communities varied in their content, class 
sizes, and in the degree to which faculty integrated 
their courses. College leaders were consistently very 
supportive of the program 


The program had a relatively high level of faculty 
collaboration, and many instructors created linked syllabi, 
visited each other’s classrooms, and discussed individual 
students’ progress. However, the degree of curricular 
integration varied among the learning communities 

Curricular integration varied, but by the second semester 
of the evaluation, more instructors collaborated on 
joint assignments and overarching themes 


Source. Scrivener et al. (2008); Visher and Teres (2011); Weiss, Visher, and Wathington (2010); Weissman et al. (2011); and 
Weissman, Cullinan, Cema, Safran, and Richman (2012). 


letters, emails, and phone calls, as well as in per¬ 
son during the registration process. Students who 
attended an intake session on campus and agreed 
to take part in the study completed an informed 
consent form and a baseline survey (described 
below). After completing these forms, each stu¬ 
dent was assigned, at random, either to the pro¬ 
gram group, whose members had the opportunity 
to enroll in learning communities, or to the control 


group, whose members had the opportunity to 
receive the college’s standard services. The ran¬ 
dom assignment decision was controlled by 
MDRC. Each college’s research sample was 
comprised of three or four cohorts of students. 
Each cohort started at the beginning of subsequent 
semesters. For example, at Hillsborough, students 
in the first cohort were randomly assigned during 
the registration period for the fall 2007 semester, 
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with the expectation that the program group 
would enroll in learning communities in fall 
2007; the second cohort of students did so in 
spring 2008, and the third in fall 2008. In addi¬ 
tion, three of the colleges included more than one 
campus in the study. The random assignment 
process occurred separately at each college (and 
campus, when applicable) for each cohort. 

Table 2 provides descriptive background 
information on the study sample at baseline, by 
research group. Breaking the data up by research 
group shows the extent that the random assign¬ 
ment procedure yielded two groups that are sta¬ 
tistically similar at baseline. 

Mirroring national trends and each college’s 
student population, the majority of study partici¬ 
pants were women. Study participants were pri¬ 
marily of traditional college age at baseline, and 
they were racially diverse—there was no racial 
majority. Most sample members did not have any 
children at the start of the study, and more than 3 
in every 10 study participants reported being the 
first person in their family to attend college. Over 
a third of the overall sample spoke a language 
other than English regularly at home. As Table 2 
shows, study participants came from a diverse set 
of backgrounds and experiences. It is worth not¬ 
ing that students’ background characteristics var¬ 
ied across the six colleges, which is one reason 
that it is interesting to examine whether program 
impacts varied among the colleges. 

Table 2 also includes p values from a chi- 
square test of independence, which provides a 
sense of whether there were significant baseline 
differences in the characteristics of program and 
control group members. As Table 2 shows, the 
program and control group were very similar at 
the start of the study. Similar baseline tables are 
available for each individual college through past 
reports (Scrivener et al., 2008; Visher & Teres, 
2011; Weiss et al., 2010; Weissman et al., 2011; 
Weissman et al., 2012). Across these six evalua¬ 
tions, random assignment generally did not yield 
systematic differences in observable baseline 
characteristics between program and control 
group members. As noted in Richburg-Hayes et 
al. (2008), Kingsborough’s sample had more sta¬ 
tistically significant differences in baseline char¬ 
acteristics than one would expect to observe by 
chance alone; however, they conducted sensitiv¬ 
ity analyses that yielded qualitatively similar 


findings and thus do not express concern with 
regard to potential bias in their impact estimator. 

The precise population to which the sample of 
students and sample of colleges generalize is diffi¬ 
cult to define due to the nonrandom sample selec¬ 
tion at both the student and school levels; however, 
the institutions include a variety of programs, loca¬ 
tions, and populations served, including nearly 
7,000 students and 174 learning communities. 

Measures 

Student Background Data. As described above, 
students completed a short baseline survey on 
background characteristics immediately before 
being randomly assigned. These data were used 
to describe the sample, to examine the similarity 
between research groups at the onset of the study, 
and to identify students for subgroup analyses. 

Student Records Data. Each college provided 
information on students’ academic outcomes 
(e.g., enrollment and credit accumulation) from 
student transcript records. This article presents 
transcript data outcomes during the “program” 
semester, when program group members were 
eligible to participate in learning communities, 
and in two subsequent “postprogram” semesters. 

Description of Key Outcomes. We focus on three 
primary indicators of student academic progress: 
credits earned in a targeted subject area (English 
or mathematics), credits earned outside the tar¬ 
geted subject area, and total credits earned. The 
credit accumulation policies at the six colleges are 
generally comparable; at all six colleges, credits 
are granted for developmental courses, but these 
credits do not count toward a degree. For students 
to earn an associate’s degree at each college, they 
must complete at least 60 college-level credits. 

Credits earned in the targeted subject 
area. The learning community programs all 
included at least one subject-specific develop¬ 
mental course, either mathematics or English, 
in their learning community link. The theory 
of change suggested that learning communities 
would help students progress through the tar¬ 
geted subject’s course sequence more quickly. 
To assess the program’s impact on this outcome, 
the number of credits attempted and earned in 
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TABLE 2 


Sample Members Baseline Characteristics by Research Group 


Program group 

Control group 

M difference 

p value 

Gender (%) 




.5410 

Male 

43.5 

42.8 

0.7 


Female 

56.5 

57.2 

-0.7 


Race/ethnicity a (%) 




.1030 

Hispanic 

34.6 

34.9 

-0.3 


White 

18.4 

17.8 

0.7 


Black 

33.5 

31.8 

1.7 


Asian or Pacific Islander 

6.6 

7.2 

-0.6 


Other b 

3.0 

4.0 

-1.0 


Missing 

3.8 

4.3 

-0.5 


Age (%) 




.3800 

20 years old and younger 

71.6 

70.1 

1.4 


21-25 years old 

16.0 

16.8 

-0.8 


26-30 years old 

5.1 

5.8 

-0.7 


31 and older 

7.3 

7.1 

0.1 


Single parent (%) 




.3650 

Yes 

13.1 

13.7 

-0.6 


No 

69.7 

70.2 

-0.5 


Missing 

17.3 

16.1 

1.2 


Has one or more children (%) 




.6540 

Yes 

18.1 

18.6 

-0.5 


No 

78.6 

77.7 

0.8 


Missing 

3.4 

3.7 

-0.3 


First person in family to attend college (%) 




.6890 

Yes 

31.9 

32.7 

-0.7 


No 

62.3 

61.9 

0.4 


Missing 

5.7 

5.4 

0.4 


Language other than English spoken 




.3580 

regularly at home (%) 





Yes 

35.7 

37.2 

-1.5 


No 

61.1 

59.5 

1.7 


Missing 

3.1 

3.3 

-0.2 


Financially dependent on parents (%) 




.3690 

Yes 

39.7 

41.1 

-1.3 


No 

43.0 

42.8 

0.2 


Missing 

17.2 

16.1 

1.1 


Highest grade completed (%) 




.2470 

11th grade or lower 

12.7 

13.5 

-0.8 


12 th grade 

83.1 

81.7 

1.4 


Missing 

4.2 

4.8 

-0.6 


Diplomas/degrees earned 0 (%) 




.4300 

General educational development (GED) 

13.9 

15.0 

-1.1 


High school diploma 

79.2 

77.4 

1.8 


Occupational/technical certificate 

4.6 

4.0 

0.6 


Two-year or higher degree 

0.4 

0.5 

0.0 



(continued) 
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TABLE 2 (CONTINUED) 



Program group 

Control group 

M difference 

p value 

None of the above 

2.1 

2.5 

-0.4 


Missing 

3.7 

3.8 

-0.1 


Taken any college courses (%) 




.4450 

Yes 

17.9 

18.8 

-1.0 


No 

78.4 

77.2 

1.3 


Missing 

3.7 

4.0 

-0.3 


Sample size (total = 6,974) 

3,983 

2,991 




Source. MDRC calculations from Baseline Survey data. 

Note. Calculations for this table used all available data for the 6,974 sample members. Random assignment ratios vary across 
cohorts. Estimates are weighted to account for probability of being assigned to the treatment group. Distributions may not add 
to 100% because of rounding. 

“Respondents who said they are Hispanic and chose a race are included only in the Hispanic category. Respondents who said they 
are not Hispanic and chose more than one race are included only in the multiracial category. 

b “Other” race/ethnicity includes those who marked "Other,” more than one race, or Native American/Alaska Native. 
‘Distributions do not add to 100% because categories are not mutually exclusive. 


the targeted subject area was measured. For 
example, at those schools targeting developmen¬ 
tal math, this measure sums the total number of 
math credits earned, including both develop¬ 
mental (which do not count toward a degree) 
and college-level credits (meaning credits that 
are degree-applicable). 

Credits earned outside the targeted subject 
area. Focusing efforts on a specific developmen¬ 
tal subject may also lead to improved progress in 
courses outside that subject area—for example, 
the skills learned in developmental English may 
lead to success in history or sociology courses. 
Conversely, focusing efforts on a targeted sub¬ 
ject may slow progress outside the targeted sub¬ 
ject if students (a) put off other required courses 
(substituting the targeted developmental course 
for a course they otherwise would have taken), 
or (b) focus their time and energy on the targeted 
subject to the detriment of performance in other 
subject areas. To assess the program’s effects on 
progress outside the targeted subject, the total 
number of credits earned (developmental and 
college level) outside the targeted subject is mea¬ 
sured. 

Total credits earned. Ultimately, the most 
important question we seek to address is whether 
learning communities improve students’ over¬ 
all progress toward a degree. There is no perfect 
measure of progress toward a degree, only useful 


proxies. The proxy used here is calculated by sum¬ 
ming together credits earned inside and outside 
the targeted subject area, including developmental 
and college-level credits. For students who place 
into developmental coursework, developmental 
courses may be required before taking certain 
required college-level courses, so these students 
must complete more than 60 credits to earn a 
degree. Generally speaking, earning more credits 
brings a student closer to earning a degree, so total 
credit accumulation serves as a very good indica¬ 
tor of overall progress toward a degree. 

Readers may wonder why persistence (enroll¬ 
ment in subsequent semesters) is not listed as a 
primary/confirmatory indicator of the program’s 
effectiveness as it is a key mechanism by which 
learning communities are theorized to work. 
While we consider enrollment a key potential 
mediating variable due to its importance as it 
relates to the theory of change of learning com¬ 
munities, we posit that enrollment without credit 
accumulation does not necessarily indicate prog¬ 
ress. Moreover, it is likely that a program that 
increases enrollment will also increase credit 
accumulation by changing unenrolled students 
into positive credit earners. Consequently, by 
examining credit accumulation we account for 
the realized observable benefits of persistence. 
The programs’ estimated impacts on enrollment 
are explored, but the main purpose of these anal¬ 
yses is to better understand one mechanism 
through which learning communities might 
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improve students’ academic progress as mea¬ 
sured by credit accumulation. 

Implementation of the Intervention 

Implementation research was conducted at all 
six colleges using a variety of observations, 
interviews, and other methods to collect data. 
College-specific findings from the implementa¬ 
tion research can be found in detail in previously 
published reports (Richburg-Hayes et al., 2008; 
Scrivener et al., 2008; Visher et al., 2010; Visher 
& Teres, 2011; Weiss et al., 2010; Weissman et 
al., 2011; Weissman et al., 2012), and are con¬ 
cisely described in column 2 of Table 1. 

To summarize, all six colleges successfully 
linked a targeted developmental subject course 
with another course (and in KCC’s case, two 
other courses ) and enrolled a group of students in 
these linked courses. Although several of the col¬ 
leges experienced initial challenges in course 
scheduling, setting up enrollment processes, and 
so on, in the end, the coenrollment of students 
into two or more courses was very well imple¬ 
mented across all colleges. Throughout the eval¬ 
uations (even for the final cohorts), there was a 
significant variation in the amount of faculty col¬ 
laboration within the colleges—often ranging 
from almost none to routine, weekly (and on 
occasion, daily) check-ins. Likewise, efforts to 
integrate course content by aligning syllabi and 
assignments across the courses varied dramati¬ 
cally among teaching teams within each college, 
although there is some evidence that this compo¬ 
nent became stronger over time at several of the 
colleges. Curricular alignment was the most 
challenging component of learning communities 
to implement consistently, and the one that was 
met with the least success. Finally, extra student 
supports were generally offered as intended, 
although support other than that provided in 
linked student success courses was fairly mini¬ 
mal (with the exception of KCC’s enhanced stu¬ 
dent services and book voucher). 

Results 

The main analyses presented here are ITT— 
comparing all program group students with all 
control group students, regardless of compli¬ 
ance. 4 The estimates reflect the effect of being 


offered the opportunity to participate in learning 
communities. This is not necessarily the same as 
the effect of experiencing a learning community 
as only 71% of program group members partici¬ 
pated in a learning community. A brief discussion 
of sensitivity analyses that were conducted 
among a subgroup of students with higher rates 
of compliance is provided at the end of the this 
section. The appendix describes the impact 
model used for analyses. 

The results are divided into four sections. The 
first section describes the pooled overall average 
effect of the opportunity to participate in learning 
communities. The second section examines 
whether the programs’ effects vary among the six 
colleges studied (as well as other groupings). The 
third section discusses the sensitivity of results to 
program participation. The final section summa¬ 
rizes the results and discusses limitations. 

Pooled Overall Average Effects 

Overall, the learning communities had a half¬ 
credit impact (positive) on students’ total credit 
accumulation. This impact was driven by credits 
earned in the targeted subject, with no discern¬ 
ible effect on credits earned outside the targeted 
subject. The effect was maintained for three 
semesters, although it did not change after the 
first (program) semester. Table 3 presents the 
overall average effects of learning communities 
pooled across the six colleges. 

Credits Earned in the Targeted Subject Area. 
Learning communities had a positive effect on 
credit accumulation in the targeted subject (math¬ 
ematics or English). During the first semester, 
program group students earned an average of 
2.72 credits in the targeted subject, compared 
with 2.20 credits for control group students. 
These credits were almost entirely developmen¬ 
tal. The 0.52 credit impact represents a 24% 
increase over the control group base (0.52 / 2.20). 
For context, most developmental courses in this 
study carried three or four credits, so the esti¬ 
mated impact represents around one sixth or one 
eighth of a typical developmental course. 

The learning communities’ estimated effect 
on credits earned in the targeted subject during 
the first semester is fairly small in magnitude. 
Nonetheless, it is conceivable that even a small 
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TABLE 3 

Pooled Academic Outcomes 


Outcome 

Program 

M 

group 

SD 

Control 

M 

group 

SD 

M difference 

p value 

In the targeted subject 3 

Cumulative credits attempted 

First semester 

4.13 

3.05 

3.57 

3.17 

0.56 

.0000 

Second semester 

5.83 

4.28 

5.23 

4.30 

0.60 

.0010 

Third semester 

6.72 

4.96 

6.19 

5.04 

0.53 

.0060 

Cumulative credits earned 

First semester 

2.72 

3.08 

2.20 

3.04 

0.52 

.0000 

Second semester 

4.02 

4.46 

3.52 

4.42 

0.50 

.0020 

Third semester 

4.69 

5.06 

4.25 

5.12 

0.44 

.0110 

Outside the targeted subject 

Cumulative credits attempted 

First semester 

6.04 

4.04 

6.31 

4.62 

-0.27 

.0440 

Second semester 

11.14 

7.72 

11.37 

8.40 

-0.23 

.3220 

Third semester 

15.44 

11.60 

15.60 

12.31 

-0.17 

.6150 

Cumulative credits earned 

First semester 

4.19 

3.92 

4.19 

4.33 

0.00 

.9940 

Second semester 

7.88 

7.53 

7.84 

8.05 

0.05 

.8340 

Third semester 

10.87 

11.00 

10.85 

11.67 

0.01 

.9660 

Total credits 

Cumulative credits attempted 

First semester 

10.18 

5.67 

9.88 

6.02 

0.29 

.0990 

Second semester 

16.99 

10.26 

16.60 

10.68 

0.38 

.2130 

Third semester 

22.18 

14.74 

21.81 

15.36 

0.37 

.3920 

Cumulative credits earned 

First semester 

6.91 

5.74 

6.38 

5.81 

0.53 

.0020 

Second semester 

11.92 

10.34 

11.36 

10.61 

0.56 

.0580 

Third semester 

Sample size (total = 6,974) 

15.58 

3,983 

14.31 

15.10 

2,991 

14.9 

0.47 

.2280 


Source. MDRC calculations from the Community College of Baltimore County, Hillsborough Community College, Houston 
Community College, Kingsborough Community College, Merced College, and Queensborough Community College transcript 
data. 

Note. “Targeted subject” credit measures and “outside the targeted subject” credit measures may not sum to “total” credit mea¬ 
sures because Hillsborough Community College includes vocational credits in total credit measures. These vocational credits are 
not included in targeted subject credit measures or in outside the targeted subject credit measures. Cumulative measures exclude 
repeated courses. Estimates are adjusted by campus and cohort. Cluster-robust standard errors are used. A two-tailed test was 
applied to estimated differences between research groups. 

a Courses in the targeted subject for Kingsborough Community College, Hillsborough Community College, the Community Col¬ 
lege of Baltimore County, and Merced College are English courses and include both college-level and developmental credits. 
Courses in the targeted subject for Houston Community College and Queensborough Community College are math courses and 
include both college-level and developmental credits. 


boost early on could grow into a larger impact 
over time. For this reason, students’ outcomes 
on this measure were tracked for an additional 
year after the program was completed to assess 
whether the impact grew, decreased, or remained 


the same. The effect was maintained through the 
third semester of study (two semesters after 
most program group students participated in 
learning communities), but it did not increase in 
magnitude. 


530 




Learning Communities for Students in Developmental Education 


Credits Earned Outside the Targeted Subject 
Area. During the first semester, learning com¬ 
munity students attempted fewer credits (-0.27) 
outside the targeted subject than their control 
group counterparts, suggesting a small amount of 
“substitution” in students’ course enrollments. 
That is, learning communities induced students 
to attempt more credits in a targeted subject area; 
however, in doing so, on average, students 
attempted slightly fewer credits in untargeted 
subjects. Despite this substitution, the average 
number of untargeted credits earned was strik¬ 
ingly similar between program and control group 
students during the first semester. This occurred 
because program group students passed their 
untargeted credits at a slightly higher rate than 
their control group counterparts. This implies 
that the substitution did not hold students back 
with respect to their academic progress outside 
the targeted subject. Notably, while most of the 
courses in the targeted subject were at the devel¬ 
opmental level, most of the untargeted courses 
were at the college level. Given the null finding 
on untargeted credits earned, it is unsurprising 
that the program also had no discernible effect on 
college-level credits alone (not shown in tables). 

After the first semester, program and control 
group students attempted and earned untargeted 
credits at a similar pace. Despite the small posi¬ 
tive effect on students’ progress in a targeted sub¬ 
ject, generally speaking, the learning community 
programs had no discernible effect (negative or 
positive) on students’ progress outside the tar¬ 
geted subject. 

Total Credits Earned. Summing together credits 
earned in the targeted and untargeted subjects 
yields students’ total credits earned. During the 
first semester, learning community students 
earned, on average, 0.53 total credits more than 
their control group counterparts. In the absence 
of learning communities, students earned an 
average of 6.38 total credits, so the increase of 
0.53 represents an improvement of 8%. This 
impact on total credits earned was a result of stu¬ 
dents earning more credits in the targeted 
subject. 

Over the two postprogram semesters, the 
magnitude of the estimated impact on total cred¬ 
its earned remained stable. The programs’ impact 
on total credits earned occurred during the first 


semester and the advantage was largely main¬ 
tained through two postprogram semesters 
(although the cumulative effect estimate was no 
longer statistically significant at the end of three 
semesters). 

Persistence 

Although not considered a primary indicator 
of program success, we provide the programs’ 
estimated effect on persistence (as measured by 
enrollment in each postprogram semester) as part 
of the theory of change underlying learning com¬ 
munities is that they will increase students’ likeli¬ 
hood of persisting, and, in doing so, improve 
future academic outcomes such as credit accu¬ 
mulation. We do not find evidence of discernible 
effects on enrollment during the first three 
semesters of study. 5 This finding helps explain 
why the impact of learning communities does not 
grow beyond the program semester. 

In sum, during the first semester of study, 
learning communities helped students earn an 
additional half credit in their targeted subject. 
This benefit did not come at the expense of prog¬ 
ress outside the targeted subject. After the first 
semester was complete, learning community stu¬ 
dents maintain the edge they gained during that 
semester, but otherwise their academic trajectory 
remains the same. After three semesters, students 
enrolled in learning communities were, on aver¬ 
age, one half of a developmental credit ahead of 
students offered their colleges’ regular services. 

Variation in Impacts by College 

The pooled average impact of learning com¬ 
munities provides an overall estimate of the aver¬ 
age effectiveness of the six programs studied. 
However, overall average impacts can mask dif¬ 
ferences in impacts across colleges, types of stu¬ 
dents, and so on. It is therefore useful to formally 
test for evidence that the learning community pro¬ 
grams’ effects were heterogeneous. Policymakers 
and other decision makers have become increas¬ 
ingly interested not only in understanding what 
works but for whom and under what circum¬ 
stances. A program whose effects vary substan¬ 
tially and unpredictably may be a risky option for 
decision makers. A program whose effects vary 
predictably may have implications for targeting 
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FIGURE 1. ITT impact estimates on credits earned in the targeted subject at the end of the first (program) 
semester, by college. 

Source. MDRC calculations from the CCBC, Hillsborough Community College, Houston Community College, Kingsborough 
Community College, Merced College, and Queensborough Community College transcript data. 

Note. ITT = intent-to-treat; CCBC = Community College of Baltimore County. 


services to identifiable subpopulations (Weiss, 
Bloom, & Brock, 2014). 

As the random assignment of students 
occurred within each college, it is possible to 
obtain an unbiased estimate of the effect of learn¬ 
ing communities at each college. Comparing the 
impact findings among colleges requires pru¬ 
dence—it is not possible to pin down the exact 
factor(s) that cause variation in impacts among 
colleges. Differences may be a result of a variety 
of factors, including student characteristics, con¬ 
text, the service contrast, or a combination of any 
of these. In addition, some variation in obsen’ed 
impact estimates is always expected due to esti¬ 
mation error. The below analyses examine 
whether the observed variation in impacts among 
colleges is greater than what is expected due to 
chance, if the average impacts were homoge¬ 
neous among the colleges. If there is no strong 
evidence that the impacts vary across colleges, 
then the pooled results described above may pro¬ 
vide a reasonable description of the average 
effect of learning communities at the six colleges 
in this study. If there is evidence that the pro¬ 
grams’ impacts vary across the six colleges, then 
there is a good reason to believe that this type of 


program is more effective at some colleges and 
less effective at others, and it would be important 
to try to understand the reasons behind those 
differences. 

Figures 1 to 3 display the estimated impacts 
on the three key outcomes of interest, for each 
college (labeled with black squares) as well as 
pooled across all six colleges (labeled with a 
white diamond). These figures provide a visual 
representation of the observed variation in the 
estimated impacts of learning communities at the 
six colleges. 

Credits Earned in the Targeted Subject Area. F ig- 
ure 1 displays each college’s estimated impact 
and 95% confidence interval on progress through 
the targeted subject during the first semester. For 
example, at Flouston, the estimated impact is 
0.43 credits, and the confidence interval indi¬ 
cates 95% confidence that the true impact lies 
between 0.22 and 0.64 credits. 

With respect to credits earned in the targeted 
subject, there is clear evidence that the effect of 
learning communities varied by college (a joint F 
test yields p < .01). While the pooled average 
impact estimate is around 0.52, there is more to 
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FIGURE 2. ITT impact estimates on credits earned outside the targeted subject at the end of the first 
(program) semester, by college. 

Source. MDRC calculations from the CCBC, Hillsborough Community College, Houston Community College, Kingsborough 
Community College, Merced College, and Queensborough Community College transcript data. 

Note. ITT = intent-to-treat; CCBC = Community College of Baltimore County. 
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FIGURE 3. ITT impact estimates on total credits earned at the end of the first (program) semester, by college. 
Source. MDRC calculations from the CCBC, Hillsborough Community College, Houston Community College, Kingsborough 
Community College, Merced College, and Queensborough Community College transcript data. 

Note. ITT = intent-to-treat; CCBC = Community College of Baltimore County. 


the story than the pooled impact estimate. At 
KCC, for example, the learning communities 


estimated impact is 1.49 credits, nearly 3 times 
the pooled average estimate; at two colleges 
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(CCBC and Hillsborough), the estimated effects 
are significantly below the pooled average. This 
suggests that, with respect to credit accumulation 
in a targeted subject, learning communities have 
the potential to have a significantly larger or 
smaller impact than the pooled estimate. Signs of 
impact variation among colleges on cumulative 
credits earned in the targeted subject remain evi¬ 
dent through three semesters (a joint F test yields 
p = .04). A discussion of the unique elements of 
KCC’s program, which may explain why KCC’s 
program appears particularly effective, is pro¬ 
vided later in the “Discussion” section. 

The six evaluations discussed in this article 
were not designed to disentangle competing 
explanations for cross-college impact variation. 
That noted, across the six colleges there is a strik¬ 
ing relationship between the programs’ estimated 
impact on credits attempted (not shown in fig¬ 
ures) and credits earned in the targeted subject— 
programs with larger impacts on credits attempted 
have larger impacts on credits earned ( r > .95). 
With only six colleges, establishing a pattern is 
difficult; but there is clearly suggestive evidence, 
which aligns with the pooled analyses, that an 
important mechanism through which learning 
communities impact credit accumulation in the 
targeted subject is by leading students to attempt 
more credits in the targeted subject. 

Credits Earned Outside the Targeted Subject 
Area. Figure 2 displays each college’s estimated 
impact and 95% confidence interval on credits 
earned outside the targeted subject during the 
first semester. There is little observed variation in 
estimated impacts across colleges on this out¬ 
come—the amount of observed variation is no 
greater than what is expected by chance (a joint F 
test yields p = .26). Across the six colleges, learn¬ 
ing communities had a fairly homogeneous aver¬ 
age effect on students’ progress outside the 
targeted subject. In other words, the pooled 
impact estimate basically tells the full story— 
there is no strong evidence that these programs 
help (or harm) students with respect to credit 
accumulation outside the targeted subject. 

Total Credits Earned. Figure 3 displays each 
college’s estimated impact and 95% confidence 
interval on total credits earned during the first 
semester. This outcome is our proxy for students’ 


overall progress. During the first semester, stu¬ 
dents offered the opportunity to participate in 
learning communities earned an average of 0.53 
credits more than similar students without that 
opportunity. In Figure 3, there is some appear¬ 
ance that the impact estimates vary by college; 
however, the amount of observed variation in 
impacts is not greater than what is expected by 
chance if the true impacts were the same across 
all colleges (a joint F test yields p = .19). 

Given that there is no strong evidence that the 
programs’ effects vary across colleges (and each 
college’s estimate is fairly precise), the pooled 
average effect provides a useful estimate of the 
average effect of learning communities at these 
six colleges—masking limited differences in 
average impacts among the colleges. It is impor¬ 
tant to be aware that the programs’ effects still 
may vary significantly within colleges based on 
student characteristics, fidelity of implementa¬ 
tion, control group services received, and so on; 
however, this analysis suggests that among these 
six colleges the programs’ average effects on 
total credits earned were fairly homogeneous. 

Additional Analyses of Impact Variation 

We also investigated whether the effects of 
learning communities varied by students’ back¬ 
ground characteristics. Two primary subgroups 
were specified prior to conducting analyses for 
this article: race by gender and recent high school 
graduate. These variables were selected chiefly 
because of their known associations with aca¬ 
demic outcomes (Green & Radwin, 2012), rather 
than because of a strong theory why learning 
communities are expected to work better for one 
group compared with another. While these back¬ 
ground characteristics did correlate with indica¬ 
tors of academic success, evidence suggests that 
the effects of learning communities did not vary 
significantly by these characteristics. 

Additional exploratory subgroup analyses 
were conducted on four subgroups that were not 
prespecified, but were probed for hypothesis 
generation: first in family to attend college, usu¬ 
ally speaks a language other than English at 
home, single parent, and parents pay for more 
than half of expenses. Across these subgroups, 
there was little evidence of discernible impact 
variation, with the exception that the program 
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appeared more effective for students who were 
financially dependent on their parents (i.e., stu¬ 
dents who reported that their parents pay for 
more than half their expenses). This result is 
intriguing, although we are hesitant to draw too 
much from this finding, given the number of 
exploratory subgroups we examined. This is a 
discovery that future researchers of learning 
communities are encouraged to explore further. 

Finally, we note that although all six colleges 
had conducted learning communities prior to the 
evaluation, every college (including those with 
the most prior experience) needed to scale up 
their developmental learning communities and 
offer more sections to participate in the study. As 
one of the main implementation findings at sev¬ 
eral colleges was that their developmental learn¬ 
ing communities improved over the course of 
scaling up during the evaluation, we explored the 
hypothesis that the effects of learning communi¬ 
ties varied within colleges between cohorts. If, 
broadly speaking, better implemented programs 
yield larger impacts, impact variation might be 
observed among cohorts as the quality of pro¬ 
gram implementation changed over time. Across 
the three key outcomes in this study, there was no 
strong evidence supporting the hypothesis that as 
the programs matured, the estimated impacts 
improved. This provides some indication that the 
fairly modest estimated impacts are likely not a 
result of programs being studied too early in their 
implementation and scaling cycle. 

Sensitivity of Results to Participation Rates 

An important consideration when examining 
program effects from a randomized experiment 
is the level of program participation. Across the 
six colleges, 71% of program group members 
were coenrolled in classes that made up a learn¬ 
ing community at the colleges’ add/drop deadline 
(suggesting that they received a significant dose 
of the program). The main analyses presented in 
this article are ITT, so they do not necessarily 
reflect the impact of experiencing the program. 
Many readers may be interested in the effect of 
the program on those who participate, or the 
treatment-on-treated. While researchers often 
estimate TOT (or local average treatment effect 
[LATE]) using instrumental variables (IVs) anal¬ 
ysis, the assumptions required for such analyses 


are likely violated in this study (Angrist, Imbens, 
& Rubin, 1996; H. S. Bloom, 1984). Specifically, 
such analyses require the assumption that the 
treatment has no impact on the “no-shows” (or 
“never-takers”), those randomly assigned to the 
program who did not experience learning com¬ 
munities. In this study, this is a strong assump¬ 
tion because program participation is defined 
several weeks into the semester, at a point when 
the program may already have affected some stu¬ 
dent’s enrollment decisions or academic out¬ 
comes. In addition, our best measure of program 
participation is defined as coenrolling in a learn¬ 
ing community link, but some colleges offered 
additional program services, which a student 
might benefit from even if not coenrolled in a 
learning community link. In other words, the 
“exclusion restriction” may not hold, so such 
analyses would rely on major assumptions. 

To come closer to estimating the effect of learn¬ 
ing communities on those students who experi¬ 
enced learning communities, without such strong 
assumptions, analyses were conducted on a subset 
of the full sample of students—those who were 
enrolled in at least one class at the add/drop dead¬ 
line of the first semester. The percentage of pro¬ 
gram and control group members enrolled at the 
add/drop deadline is statistically indistinguish¬ 
able, so it is reasonable to expect that these groups 
were similar at the outset of the study (the weaker 
assumption required for these analyses to be inter¬ 
nally valid). This subset of students is of interest 
because they had a significantly higher rate of pro¬ 
gram take-up (84%, compared with 71% for the 
full sample). As a result, analyses using this subset 
of students provide a closer estimate of the effect 
of the TOT. These sensitivity analyses confirm the 
pooled overall findings and result in almost no 
substantive changes in the magnitude of the 
pooled overall average impact estimates. This pro¬ 
vides some evidence that the pooled overall analy¬ 
ses presented in this report are not too different 
than the effect of learning communities on those 
who participate in them. 

Unlike the ITT analyses that suggested the 
programs’ effect on total credits earned were 
fairly homogeneous across colleges, the sensitiv¬ 
ity analyses suggest that the programs’ effects do 
vary across colleges on total credits earned dur¬ 
ing the first semester. However, by the end of the 
third semester, there is again not meaningful 
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evidence of variation in impacts among colleges 
on total credits earned. 

Limitations 

Before discussing the implications of the find¬ 
ings, we would like to point out a few important 
limitations to this research. 

Limited External Validity. This study examines 
six learning community programs implemented 
at six community colleges. The randomized 
design offers a major improvement over past 
studies with respect to internal validity (for a 
definition, see Campbell & Stanley, 1963). We 
attempted to select a diverse group of colleges, 
operating in different contexts, serving diverse 
types of students, with programs representing a 
range of semester-long developmental education 
learning communities. However, the study was 
conducted on a nonrandom sample of six com¬ 
munity colleges, so inferences regarding the 
impacts of learning communities at other col¬ 
leges require speculation. Also, developmental 
learning communities at the colleges were 
offered with somewhat limited schedules; for 
example, very few learning communities were 
offered as night courses, so students only avail¬ 
able for courses at these hours were unlikely to 
be represented in this sample. 

Presence of Teacher Effects. In this study, ran¬ 
dom assignment was conducted at the student 
level, creating two similar groups of students for 
future comparisons. Due to practical restrictions, 
instructors were not randomly assigned; thus, the 
selection of instructors may have influenced the 
estimated effects of learning communities for 
better, for worse, or not at all. The presented 
effect estimates represent the combined effect of 
learning communities and the types of instructors 
delivering those services, or teacher effects, 
whereas we are most interested in the effects of 
learning communities alone (Weiss, 2010). To 
some extent, this concern is mitigated as many 
learning community instructors also taught 
stand-alone versions of these classes, sometimes 
to control group students. However, it is unclear 
how frequently control group students took 
classes with instructors who also taught the pro¬ 
gram group. 


To the extent that the instructor selection 
mechanisms varied across colleges and cohorts 
and yet the impact estimates remained fairly con¬ 
stant, this may increase confidence that the 
effects described are primarily caused by the 
learning communities. Nonetheless, this limita¬ 
tion is real and noteworthy. It is also noteworthy 
that this concern exists in all past evaluations of 
learning communities, so the elimination of stu¬ 
dent selection bias through the randomized 
design presented in this article is a major 
improvement over past attempts to estimate the 
effects of learning communities, even with this 
limitation. 

Learning Outcomes Not Measured. Finally, 
another limitation of this study is that we do not 
have any standardized measures (e.g., achieve¬ 
ment tests) of student learning apart from course 
completion. This is a common limitation in most 
postsecondary education research. Nevertheless, 
as college certificates and degrees are awarded 
based on credit accumulation, the academic 
progress indicators used in this study are out¬ 
comes that are of critical importance to college 
administrators, policymakers, and researchers. 

Discussion 

Learning communities have become a popular 
strategy to improve the educational trajectories 
of students who are referred to developmental 
education. This article addresses the question of 
whether this intervention leads to better aca¬ 
demic progress when compared with “business- 
as-usual” for students with developmental needs. 

On average, the single-semester learning 
communities operated in the six community col¬ 
leges in this study had a small positive effect on 
credit accumulation. This impact was achieved in 
the semester in which students enrolled in the 
learning communities, and it was driven by the 
impact of learning communities on credits accu¬ 
mulated in a targeted developmental subject area. 
In contrast to prior nonexperimental studies and 
the learning communities’ theory of change, this 
study found that these programs did not have a 
discernible effect on persistence. 

The null findings of learning communities’ 
effect on persistence may give readers pause 
regarding the relationship between integration 
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and retention in community colleges. MDRC’s 
learning communities’ evaluations were not 
designed to definitively answer questions about 
causal mediation, which may be more difficult to 
answer than is generally acknowledged (Bullock, 
Green, & Ha, 2010). It is possible that the learn¬ 
ing communities in this study increased students’ 
feelings of integration, but that increased integra¬ 
tion alone may not be sufficient to meaningfully 
change student success rates; this seems reason¬ 
able given the multiple barriers faced by many 
community college students (e.g., financial, bal¬ 
ancing the demands of school, work, and family). 
Alternatively, these single-semester learning 
communities may not have provided the addi¬ 
tional sense of integration needed to make a dif¬ 
ference in community college students’ likelihood 
of persisting. 

Because of the large sample size, we are able 
to examine whether program effects varied 
across colleges. The results of these analyses 
were mixed—there is clear evidence that the ITT 
impact estimates vary on targeted credits earned 
(with the impact at KCC being the largest), no 
evidence of impact variation on untargeted cred¬ 
its earned, and no evidence of discernible impact 
variation on total credit accumulation. Sensitivity 
analyses on a subset of students with higher par¬ 
ticipation uncover evidence of short-term varia¬ 
tion in impacts among colleges on total credits 
earned, where again KCC has the largest esti¬ 
mated impact. 

While long-term follow-up (beyond three 
semesters) data are unavailable for five of the six 
programs presented here, intriguingly, a recently 
released report on the effects of the program at 
KCC found that the early effects were sustained 
through 6 and 7 years (Sommo, Mayer, Rudd, & 
Cullinan, 2012; Weiss, Mayer, et al., 2014). 

Given that there is no discernible evidence of 
cross-college impact variation on cumulative 
total credits earned after three semesters, it is 
debatable what to make of the appearance that 
KCC’s program has larger short-term effects, and 
the knowledge that KCC’s program has long¬ 
term impacts. The qualitative research conducted 
across these six colleges pointed out several 
unique features of the KCC program, which 
could explain the larger estimated effects at 
KCC. These include the fact that (a) KCC’s 
model included more enhancements in student 


services (e.g., counseling, tutoring); (b) students 
in KCC’s program were actively encouraged to 
enroll in 6-week “intersession” (i.e., winter/sum¬ 
mer), which accounted for around one fourth of 
the 6-year impact on credits accumulated at 
KCC; (c) the sample at KCC was somewhat 
unusual for community colleges—students were 
more likely to be full-time and financially depen¬ 
dent on their parents—that is, they were more 
“traditional.” These types of students may bene¬ 
fit most from the extra support and structured 
cohort experience; this extra scaffolding of the 
transition into college may have served KCC’s 
student population particularly well; (d) the KCC 
program linked three courses rather than two; 
and (e) KCC’s program enjoyed unusually strong 
support from college administrators. Caution is 
required when trying to link these program fea¬ 
tures with the larger estimated impacts. The evi¬ 
dence is inconclusive on whether the effects truly 
vary across colleges—KCC’s larger impact esti¬ 
mate could simply be a result of estimation error. 
In addition, even if KCC’s program truly has 
larger effects than the other colleges, there are 
many plausible explanations for such differ¬ 
ences; the above listed explanations are among 
them. 

Conclusion 

It is encouraging that, on average, the learning 
community programs benefited students in 
developmental education in the six community 
colleges studied. However, the impacts were 
small by most standards, and there was no dis¬ 
cernible evidence that learning communities 
improved persistence—a key mechanism through 
which learning communities are hypothesized to 
benefit students. These findings may raise ques¬ 
tions about the viability of learning communities 
as a key strategy for improving student outcomes 
at community colleges. 

Advocates of learning communities may 
rightly point out that the learning community pro¬ 
grams that were studied here were not the “ideal” 
learning communities where every faculty pair 
collaborates on an ongoing basis and the curricula 
are tightly integrated around a well-developed 
theme—it is possible (although untested) that 
such learning communities are very effective. 
Detractors of learning communities may rightly 
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point out that this study examined the average 
effects of a variety of learning community models 
implemented across a diverse set of students, and 
the study found limited evidence that the pro¬ 
grams’ effects varied, so perhaps the small esti¬ 
mated impacts are what can be expected by this 
type of intervention. Both of these views seem 
defensible, and we do not think they are 
inconsistent. 

This research increases significantly the evi¬ 
dence which we hope college faculty and admin¬ 
istrators, policymakers, and funders will use to 
improve student outcomes at community col¬ 
leges. Evidence of the effectiveness of extremely 
well and consistently implemented learning com¬ 
munity programs (i.e., unlike those studied here) 
remains limited, although through our imple¬ 
mentation research it appears that achieving high 
levels of depth and breadth is a major challenge, 
and probably quite rare. Given the current evi¬ 
dence base, to the best of our knowledge learning 
community programs with well-implemented 
coenrollment and significant variation in imple¬ 
mentation of the other key components have a 
small but positive benefit to students. 

Appendix 

To estimate the effect of being offered the oppor¬ 
tunity to participate in learning communities, we use 
a generalized linear model including block dummy 
indicators (one for each unique college by cohort com¬ 
bination) as described by the following equation: 

Yij -« / •• [YTij-i'.ij, (1) 

where Yy is an outcome for student i in random 
assignment block j . College-by-cohort fixed effects 
are denoted by ay , and Ty is a treatment indicator 
set equal to 1 if the student was randomly assigned 
to the program group and 0 otherwise. Conserva¬ 
tively, cluster-robust standard errors are estimated 
to account for the potential lack of independence of 
errors caused by clustering in learning communities 
in the treatment arm (Weiss et al., 2010). Weights are 
used to account for varying random assignment ratios 
within college. Within each of the six colleges, the 
weights sum to the colleges' total sample size, so that 
the pooled impact estimates are roughly weighted by 
the colleges’ sample size. Weights were obtained sepa¬ 
rately for each college and created to ensure that across 
blocks within colleges the random assignment ratio is 
constant. For ease of interpretation and presentation, 


tables and figures present the least squares means for 
program and control group members. College-by¬ 
college impact estimates were obtained through the 
use of treatment-by-college interaction terms. Simi¬ 
larly, subgroup analyses were conducted through the 
use of interaction terms. 
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Melinda Gates Foundation, the Ford Foundation, 
Lumina Foundation for Education, and the Robin 
Hood Foundation. 

3. In addition to scoring higher on the English 
placement test, the estimated effect of the program 
was larger for students deemed college-ready based on 
Kingsborough Community College’s (KCC) English 
placement test (Scrivener et al., 2008). 

4. Students who did not enroll are treated as hav¬ 
ing earned zero credits that semester. This could, theo¬ 
retically, bias the impact estimator if program group 
members and control group members earned credits 
at different rates at nonstudy colleges. We believe that 
this concern is minimal because the programs did not 
have an effect on enrollment at the study colleges. 
Without an enrollment effect at the study colleges, a 
somewhat elaborate story must be concocted as to why 
the opportunity to participate in learning communities 
would nonetheless have a positive (or negative) effect 
on enrollment at other nonstudy colleges (i.e., Why 
would learning communities not effect enrollment at 
the initial college, yet have an effect on enrollment 
elsewhere?). 

5. Estimated effects on enrollment are 1.2, 1.3, 
and -0.3 percentage points in Semesters 1, 2, and 3, 
respectively. Control group enrollment rates are 84.1, 
63.8, and 49.9, and standard error estimates are 1.2, 
1.3, and 1.4, respectively. 
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